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(57) Real-time multimedia services are transmitted 
over a hybrid network including a nonguaranteed quality 
of service packet switched local area network and a cir- 
cuit switched ISDN wide area network having a central- 
ized multimedia bridge located within the wide area 
network The local area networks and multimedia 
bridge are interconnected via ISDN routers. An algo- 
rithm executed by the multimedia bridge receives sig- 



nals from the packet switched network and detects the 
absence of properties needed for real-time audio visual 
services. The data signals are processed to compen- 
sate for the absence of the properties and then are 
transmitted over the wide area network to enable real- 
time audio visual services. 
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Description 

Technical Field 

This invention relates to multimedia conferencing 
services and more particularly relates to such services 
provided over multiple nonguaranteed quality of service 
local area networks in geographically dispersed loca- 
tions interconnected by an ISDN wide area network in 
which bridging is performed in the ISDN network. 

Background of the Invention 

Real-time multimedia services have been provided 
in the past by connecting signals over circuit switched 
networks having guaranteed quality of service. 
Although this approach results in reasonable quality, it is 
cost prohibitive for many potential users. Most offices 
desiring multimedia services, such as video conferenc- 
ing, are served by a packet switched local area network 
(LAN) providing nonguaranteed quality of service. Such 
networks are inherently unreliable for multimedia. The 
multimedia packet streams coming from such networks 
do not have the proper sequencing. The arrival time 
between the packets may have significant variations 
known as delay jitter, and some packets may be lost. 
Moreover, the recovered bit streams of audio and video 
do not maintain required temporal relationships to 
insure both intramedia and intermedia synchronization 
so that lip-synchronization is maintained. Since the cost 
of installing a guaranteed quality of service network in 
such offices is prohibitive, the adoption of multimedia 
services in most offices has been severely restricted. 

One attempt to provide video and audio services is 
described in U.S. Patent No. 4,866,704 (Bergman, filed 
March 16, 1988). Bergman describes the conversion of 
data to T1 frames and then an additional conversion to 
packets for distribution on a wide band fiber optic ring. 
This arrangement is contrary to the existing local area 
network structure found in most offices, and therefore 
would required the installation of new networks at pro- 
hibitive expense. 

Another approach to transmitting data from one 
location having a local area network to a second loca- 
tion having a local area network via a T1 network is 
described in U.S. Patent No. 5,384,835 (Wheeler et al., 
filed January 12, 1993). Wheeler et al. teach such a net- 
work arrangement for distributing text data and still 
graphic data. However, they do not describe any 
method or apparatus for providing real time video and 
audio over the local area and T1 networks. 

Summary of the Invention 

My invention enables real-time multimedia confer- 
encing services to be provided to dispersed locations 
interconneted by a wide area network from multimedia 
signals originating at the locations. The multimedia sig- 



nals are audio, video or data signals, or any combina- 
tion of such signals. The locations employ 
nonguaranteed quality of service packet switched local 
area networks which degrade one or more properties 

5 required by the multimedia signals. In such an environ- 
ment, multimedia conferencing signals may be provided 
by receiving the multimedia signals from the local area 
networks at the wide area network bridging the multi- 
media signals in the wide area network and transmitting 

w the bridged multimedia signals over the wide area net- 
work to the various locations. By using this method, the 
bandwidth required to transmit the multimedia signals 
between the dispersed locations is reduced. 

According to another aspect of my invention, the 

75 absence of the required properties is detected in the 
wide area network. The multimedia signals then are 
processed to compensate for the absence of the prop- 
erties. The processed multimedia signals are transmit- 
ted to the dispersed locations over the wide area 

20 network in order to improve the rgyrtimedia services. 

My invention also may be enhanced by detecting 
the absence of the properties in the multimedia signals 
at the dispersed locations and processing the multime- 
dia signals to compensate for the absent properties. 

25 

Brief Description of the Drawings 

In the drawing, 

30 Figure 1 illustrates a preferred form of end-to-end 
network configurations for multipoint multimedia 
conferencing services through interconnection of 
nonguaranteed quality of service (NGQOS) local 
area networks (LANs) to an integrated services dig- 

35 rtal network (ISDN) via ISDN routers in accordance 
with my invention; 

Figure 2 depicts a preferred form of high-level end- 
to-end protocol architecture for the multipoint multi- 
media conferencing services through interconnec- 
40 tion of the .nonguaranteed quality of service 
(NGQOS) local area networks (LANs) to the inte- 
grated services digital network (ISDN) via ISDN 
routers in accordance with my invention; 
Figure 3 shows a preferred form of nonguaranteed 
45 quality of service (NGQOS) local area networks 
(LANs) based multimedia workstation/personal 
computer (MMWS/PC) protocol architecture in 
accordance with my invention; 
Figure 4 shows a preferred form of switched non- 
50 guaranteed quality of service (NGQOS) local area 
network (LAN) hub based multipoint multimedia 
bridging (MMB) protocol architecture in accordance 
with my invention; and 

Figures 5 through 8 show flow charts of a preferred 
55 form of algorithm for intra and inter-media synchro- 
nization to improvement performance of the real- 
time audio and video traffic in accordance with my 
invention. 
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Detailed Description 

Figure 1 illustrates multimedia work stations or per- 
sonal computers 1-1, 1-2, 1-3, 1-4 and 1-5 that are 
equipped with multimedia conferencing application pro- 
grams based on the H.323 standard of the International 
Telecommunications Union (ITU). Computers 1-1 and 
1 -2 are connected to a shared nonguaranteed quality of 
service local area network 2-1 at a customer premises 
or location L1 . Similarly, computers 1 -4 and 1 -5 are con- 
nected to a shared nonguaranteed quality of service 
local area network 2-2 located at another customer 
premises or location L2 which may be displaced from 
location L1 by many miles. Networks 2-1 and 2-2 are 
equipped with a gatekeeper function that controls the 
total loading on the network as described in ITU Rec. 
H.323. A conventional ISDN router 3-1 is located at 
location L1 for connecting signals from network 2-1 to a 
conventional circuit switched narrow band ISDN wide 
area network (WAN) 5 over a link 4-1 . Signals are proc- 
essed in time division multiplexing fashion over the 
ISDN links 4-1 through 4-4 before being transmitted 
over network 5. Similarly, a conventional router 3-4 at 
location L2 is connected to network 5 over a fink 4-4. 
ISDN routers 3-1 and 3-4 preferably have a gatekeeper 
functionality. 

Still referring to Figure 1 , a single multimedia work- 
station or personal computer 1-3 is connected to a 
switched local area network hub 5-2 at another cus- 
tomer premises or location L3 which may be displaced 
from locations L1 and L2 by many miles. Such an 
arrangement has superior performance when com- 
pared with networks 2-1 and 2-3, especially for multime- 
dia traffic. 

Networks 2-1 and 2-2, as well as hub 5-2, can be an 
Ethernet (IEEE 802.3), token ring (IEEE 802.5), fast 
Ethernet (IEEE 802.10), or FDDI (operating in the non- 
guaranteed quality of service mode). 

Still referring to Figure 1 , a multimedia bridge 6 is 
connected to a decficated high speed nonguaranteed 
quality of service local area network hub 5-1 that pro- 
vides protocol transparency with networks 2-1 , 2-2 and 
hub 5-2. Hub 5-1 is connected to multimedia bridge 6 to 
provide a guarantee of performance parameters, such 
as delay jitter, packet loss, error rate and delay. In a 
multipoint telephone call, a bad source may effect the 
entire conference. Bridge 6 detects the bad source and 
communicates with end computers 1-1, 1-2, 1-3, 1-4 
and 1-5. Bridge 6 also takes specific actions agreed 
upon at the time of call setup. The communication mes- 
sages between the computers on customer premises 
L1 - L3 via ISDN wide area network 5 insures quality of 
service for multimedia conferencing. 

ISDN routers 3-1 , 3-2, 3-3 and 3-4 are connected to 
local area network 2-1 . hubs 5-1 and 5-2 and local area 
network 2-2, respectively. The multimedia conferencing 
applications running on computers 1-1 through 1-5 are 
made in accordance with the ITU Rec. H.323 standard. 



ISDN routers 3-1 through 3-4 are connected to network 
5 via ISDN links 4-1 through 4-4, respectively. 

Multimedia bridge 6 is equipped with a multipoint 
control unit (MCU) functionality in accordance with the 

5 ITU Rec. H.323 standard, and has a general conference 
control and multipoint communication service function 
operating in accordance with the ITU Rec. T.120 stand- 
ard, as well as a multipoint controller and multiproces- 
sor function operating in accordance with the ITU Rec 

io H.323 standard. Bridge 6 also has a gatekeeper func- 
tion, such as address translation, admission control, 
bandwidth control, call control signaling, call authoriza- 
tion, bandwidth management, and call management as 
specified in the ITU Rec H.323 standard. The communi- 

15 cation between computers 1-1 through 1-5 and the 
bridge 6 takes place in a master-slave relationship as 
described in ITU Rec H.323. Bridge 6 plays the role of 
the master in controlling participating computers 1-1 
through 1-5. 

20 Bridge 6 depacketizes all encapsulated packets of 
the multimedia bit streams of signals that originate from 
computers 1-1 through 1-5 and provides bridging for 
audio, video, and/or data. Thus, in this specification and 
claims, multimedia signals means audio, video or data 

25 signals, or any combination of such signals. A point-to- 
point call initially can be processed by bridge 6 if the end 
stations (i.e., computers 1-1 through 1-5) want certain 
services, such as directory services, admission control, 
or address translation at the time of call setup, while the 

30 actual bearer channel is established via an optimal 
path. 

After the packets received from computers 1-1 
through 1 -5 have been depacketized, bridge 6 executes 
an algorithm (Figures 5-8) which serializes the packets. 

35 produces dummy packets for lost packets if appropriate, 
compensates for delay jitter where necessary and 
insures real-time audio and video synchronization as 
long as the end stations operate within certain perform- 
ance guidelines. Bridge 6 then performs audio, video 

40 and data bridging in accordance with predefined 
schemes that are agreed upon at the time of call setup. 
The bridged audio, video and data bit streams again are 
sent to the destination end stations via ISDN routers. 
The ISDN routers perform IP multicasting in sending the 

45 multimedia traffic if necessary. As an alternative to IP 
multicasting, the ITU Rec T.125 multipoint communica- 
tion service (MCS) also can be used. 

The protocol architecture for end-to-end communi- 
cations for multimedia conferencing involving computer 

so 1-1 and multimedia bridge 6 is explained in connection 
with Figure 2. The protocol stack 13-1 running on com- 
puter 1-1 has the entities GCC. MCS. H.225.0/RTP, 
TCP. UDP. IP, Logical link control (LLC), and medium 
access control (MAC) between the media stream 

55 (audio, video, and/or data) and a physical layer. Simi- 
larly, protocol stack 13-2 of multimedia bridge 6 has the 
entities GCC. MCS. H.225.0/RTP, TCP, UDP, IP, LLC. 
and MAC between the media stream (audio, video 
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and/or data) and a physical layer. ISDN routers 3-1 and 
3-2 communicate between computer 1 -1 and bridge 6 
using internet protocol (IP) (14-1, 14-2) via nonguaran- 
teed quality of service local area network 2-1 and net- 
work hub 5-1. Network 2-1 and hub 5-1 have protocol 5 
layers LLC and MAC (1 7-1 , 1 7-2). ISDN routers 3-1 and 
3-2 encapsulate the IP over the point-to-point protocol 
(PPP or the point-to-point multipoint (MP)) to transfer 
audio, video an-or data over ISDN links 4-1 and 4-2 to 
communicate over the ISDN wide area network 5. 10 
Router 3-1 is connected to a circuit switching ISDN 
node of network 5. 

The detailed architecture of computers 1-1 through 
1-5 is shown in Figure 3. The communication between 
the upper layer applications 100 (e.g., audio, video, 15 
data, system control, and/or other services ) and GCC 
106, MCS 107, or transport interface 101 can take place 
directly as needed. However, GCC 106 communicates 
with the transport interface via MCS 107 only in accord- 
ance with the ITU Rec. T.120 standard. Transport inter- 20 
face 101 makes the upper layer entities transparent 
from the details of the lower layers. Transport interface 

101 can be WinSock Forum's WINSOCK 2, Multimedia 
Communication Forum's (MMCF) TSI, or other inter- 
faces. 25 

The upper layer applications 1 00 may contain audio 
100-1. video 100-2, data 100-3, and/or other services 
100-4. Audio 105 and video 103 coming from upper 
layer applications 100 or lower layer entity H.225.0/RTP 
108 are synchronized into an entity known as synchro- 30 
nization 104. Both intramedia and intermedia synchro- 
nization services are performed by entity 104. The 
reference master time clock to provide synchronization 
is received from ISDN WAN 5 based bridge 6. The LAN 
based audio and video maintains synchronization by 35 
receiving clock signaling from the guaranteed quality of 
service ISDN WAN based bridge 6. No performance 
degradation occurs tor any media stream when it is 
transferred over ISDN WAN 5. A synchronization algo- 
rithm in accordance with this invention (Figures 5-8) 40 
maintains synchronization for different media. Comput- 
ers 1-1 through 1-5 are free to use their own synchroni- 
zation methods. However, bridge 6 uses the 
synchronization algorithm described in connection with 
Figures 5-8. 45 

Audio 105 and video 103 bit streams coming out of 
the upper layer applications are packetized in accord- 
ance with the ITU Rec. H.225.0 protocol in entity 108. 
H.225.0 is very similar to the lETPs RTP protocol. 
There can be almost direct mapping between H.225.0 so 
and RTP. The implementation details of this mapping 
function are not a part of this invention. The audio and 
video packets encapsulated in H.225.0 are transferred 
using lETPs UDP protocol in entity 109. Similarly, data 

102 and system control 106 traffic are encapsulated 55 
over the lETPs TCP protocol in entity 1 10. The system 
control traffic entity 106 includes control, call control, 
and RAS (Registration, Administration, and Status) as 



6 

specified in the ITU Rec. H.245 and H.225.0 protocols. 
However, RAS traffic is transferred using the UDP pro- 
tocol. Both UDP and TCP packets again are transferred 
using lEFTs IP protocol in entity 111. The IP packet is 
then transferred over the LAN (e.g., LAN 2-1) using the 
LAN protocol LLC in entity 112 and LAN protocol MAC 
in entity 113. 

If the packets come from the LAN to the computer 
(e.g., LAN 2-1 to computer 1-1), a similar de-encapsula- 
tion of packets takes place from MAC in entity 113 to 
LLC in entity 112, from LLC in entity 1 12~to IP in entity 
111, from IP in entity 1 1 1 to UDP in entity 1 09 or to TCP 
in entity 110. and from UDP in entity 109 to the H.225.0 
protocol in entity 108. Audio 105 and video 103 bit 
streams are recovered from the H.225.0 protocol in 
entity 108. The intermedia and intramedia synchroniza- 
tions are performed for the recovered bit streams in 
entity 104. The synchronization services in the upper 
layer (above the transport interface 101) can also be 
provided, but this service is not part of this invention. 

The protocol architecture for bridge 6 is depicted in 
Figure 4. This architecture is similar to what has been 
shown in Figure 3. Bridge 6 has the MCU and gate- 
keeper (GK) functionalities. The MCU functions of 
bridge 6 include bridging for audio, video and/or data 
similar to the functions stated for multipoint controller 
(MC) and multipoint processor (MP) of the ITU Rec. 
H.323 standard. The GK functions of bridge 6 include 
address translation, admission control, bandwidth con- 
trol, call control signaling, call authorization, bandwidth 
management, call management, directory services, and 
others. However, bridge 6 works as the master for com- 
puters 1-1 through 1-5 connected to the LANs in the 
customer premises (as shown in Figure 1) from system 
controlling point of view. Bridge 6 is connected to ISDN 
router 3-2 via a dedicated LAN hub (as shown in Figure 
1) and provides a guarantee for all performance param- 
eters. The packets coming to bridge 6 from the LANs 2- 
1 and 2-2 are de-encapsulated from MAC in entity 213 
to LLC in entity 212 to IP in entity 21 1 to UDP in entity 
210 or TCP in entity 209 as explained earlier. Both intra- 
media and intermedia synchronization for audio 207 
and video 205 streams recovered from the H.225.0 
packets in entity 208 are performed in an entity known 
as synchronization 206. A synchronization algorithm 
described in Figures 5 through 8 improves performance 
specifically targeted for the real-time traffic coming out 
of the LANs 2-1 and 2-2 and hub 5-2. Data, system con- 
trol, and other bit streams de-encapsulated from the 
TCP packets in entity 209 are sent to the upper layer 
services. 

Audio 207 and video 205 bit streams then are trans- 
ferred to the higher layer services to perform media 
bridging for audio 200-1, video 200-2, and data 200-3 
bridging similar to specifications defined in the ITU Rec. 
H.323 standard. Every conferee sets up the communi- 
cation for multipoint multimedia conferencing via bridge 
6. A point-to-point communication flow is set up 
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between bridge 6 and each end station or end system 
participating in the conference (e.g., computer 1-1 and 
1 -4). Audio bridging and video bridging are performed in 
entities 200-1 and 200-2 in accordance to the criteria 
agreed upon by the participating parties. For example, 
bridge 6 can provide either video switching or video mix- 
ing. Video switching is the process of selecting the 
video that bridge 6 outputs to computers 1 -1 through 1 - 
5 from one source to another. The criteria used to make 
the switch may be determined through detection of 
change in speaker (sensed by the associated audio 
level) or though control according to the H.245 stand- 
ard. Video mixing is the process of formatting more than 
one video source into a video stream that bridge 6 out- 
puts to computers 1-1 through 1-5. The details of the 
mixing criteria by the bridge 6 is not a part of this inven- 
tion. Bridge 6 can prepare N audio outputs from M audio 
inputs by switching, mixing or a combination of these. 
Audio mixing requires decoding the input audio to linear 
signals (pulse coded modulation or analog), performing 
a linear combination of the signals, and recording the 
result to the appropriate audio format. The details of 
such audio mixing also are not a part of this invention. 
Bridge 6 processes data in accordance with the ITU 
Rec. T.120 standard, and has functions like GCC in 
entity 201 and MCS in entity 202 as shown in Figure 4. 

Figures 5 through 8 describe a preferred form of 
algorithm to provide intra- and inter-media synchroniza- 
tion to improve performance for the real-time audio and 
video traffic over the networks shown in Figure 1 . The 
algorithm detects the absence of properties needed for 
multimedia conferencing, such as whether appropriate 
audio and video packets have been received, whether 
such packets are synchronized, and whether a com- 
plete video frame is ready. The algorithm is executed by 
synchronization entity 104 of bridge 6 (Figure 3). The 
process begins at step 500. while the initialization is 
done in step 501 . The variables shown in step 501 rep- 
resent the following: 



i.j.k = 


dummy variables representing an integer 




number 


M = 


Total number of audio packets in a given 




audio segment 


N = 


Total number of video packets in a given 




video frame 


F = 


Total number of video frames per second 


B = 


Total number of bits per video frame 


S = 


Total number of audio samples per sec- 




ond 


G = 


Total number of bits per audio sample 


T = 


Playout time 


Li = 


Reference clock time 


Aeb(k) = 


Total number of bits in audio segment k 


V to (k) = 


Total number of bits in video frame k 


♦T = 


A fixed increment in playout time 


Vth = 


A fixed threshold value for the inter-arrival 




time of the video packet 



A^ = A fixed threshold value for the inter-arrival 

time of the audio packet 
Audio packet of i-th time interval 
Video packet of i-th time interval 
Inter-arrival time of the j-th video packet 
Inter-arrival time of the i-th aucfio packet 
Average inter-arrival time for the video 
packet 

Average inter-arrival time for the audio 
packet 

Initial system time 
Final system time 
New playout time 

The measurement of inter-arrival time for audio and 
video packets for a given playout time starts at the same 
time in step 501. The average inter-arrival times for 
video and audio packets for each source are estimated 
as shown in steps 503 and 502, respectively: 

N 

V a = (1/N) £Dj 

i-i 

M 

A a = (1/M) 



In step 504, the average inter-arrival time of the 
video packet for given playout time is compared to that 
of the threshold value of the video packet inter-arrival 
time. If V a is greater than V^, bridge 6 notifies the 
source as indicated in step 506. Obtaining the notifica- 
tion from bridge 6, the source checks whether the 
shared LANs, switches, ISDN routers, or the equipment 
are degrading the performance. The source tries to rec- 
tify the problems or accepts the degraded mode of oper- 
ation. In step 505, audio packet inter-arrival time also is 
compared and actions similar to those described for 
steps 504 and 506 also are taken as stated in step 507 
if A a is greater than A^. 

In step 508, the new playout time P is estimated by 
comparing the values of T and V a . The maximum, of the 
two values is chosen as new playout time P. In step 509, 
system initial time Sj is set to reference clock time Lj that 
is being maintained by ISDN WAN 5. System final time 
is set by incrementing the system initial time by the 
playout interval: S f = S f + P . 

In step 510, the total bits for audio segment A sb (k) 
and video frame V to (k) within the given playout time are 
estimated: A sb (k) = SGP and V^fkJ^FB. It is 
expected that the playout time will be very close to one 
video frame time. Audio and video are then processed 
simultaneously through a series of steps described in 
Figures 6 and 7 as indicated in steps 511 and 512, 
respectively. 

In step 601 (Figure 6), the algorithm examines 



V p (i) = 

5 Dj = 



A a = 

10 

Si = 
Si = 
P = 



15 



20 



25 



30 



35 



40 



45 



50 
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whether all audio packets, [A p (i), i = 1 , M ] are received 
within system final time, Sf. If all packets are not 
received within the scheduled playout time, dummy 
packets are created to replace the unreceived packets 
(step 602). The creation of the dummy packets mini- 5 
mizes errors as far as practicable. Audio segment, 
Agt^k) then is formed (step 607). 

In step 603, the algorithm examines whether any 
audio packets received in step 601 do not belong to 
audio segment. A^k). The packets that do not belong w 
to this audio segment are pre-buffered (step 605) if the 
packets could be used in the next segment. In step 607, 
audio segment A^k) is created only with those packets 
that belong to that segment In step 61 0, the audio proc- 
ess, after creation of the segment, follows a series of is 
steps as indicated in Figure 8. 

Similarly, in step 701 of Figure 7, the algorithm 
examines whether all video packets, [V p (j), j = 1, N] 
are received within system final time, S f . If all packets 
are not received within the scheduled playout time, sys- 20 
tern final time S t is incremented by a small fraction of 
playout time (step 702) VT to set the new system final 
time: 

S f = S f + +T. 25 

On the other hand, if all video packets are received 
within system final time S f , the algorithm examines 
(step 703) whether any video packets not belonging to 30 
video frame V to (k) are received. If no excess packets 
are received, the video process is sent to step 805 of 
Figure 8 to follow a series of steps as indicated in step 
71 2. However, if there are excess video packets in video 
frame V^k), all excess video packets are dropped (step 35 
706), and the process is sent to step 802 of Figure 8 as 
indicated in step 709. 

In step 704, the algorithm again examines whether 
all video packets, [Vp (j), j = 1, N] are received within 
new system final time, S f . If there are excess video 40 
packets in, video frame, V to (k) the excess video packets 
are dropped (step 705) and the process is then sent to 
step 802 of Figure 8 as shown in step 709. If all video 
packets are not received within the new system final 
time, the process is sent to step 707. 45 

In step 707, the algorithm examines whether 
dummy video packets can be created to fill up the gaps 
of video frame, Vfbfk), with minimum errors. If it is possi- 
ble to create video frame V to (k) using dummy packets, 
the process is then sent to step 801 of Figure 8 (step so 
713). However, all video packets [V p (j), j = 1, N] of 
video frame V te (k) are dropped (step 708), if this frame 
cannot be created due to non-availability of the video 
packets, and the process is sent to step 802 of Figure 8 
as shown in step 709. 55 

In step 800 of Figure 8, the audio process (from 
step 610 of Figure 6) joins to the video process in step 
806, 81 2 or 810 depending on the video events for inter- 



media synchronization between the audio segment and 
video frame for playout, storage, or processing as 
needed by the multimedia applications. 

In step 810, a video frame (from step 712 of Figure 
7) and an audio segment (from step 61 0 or Figure 6) are 
synchronized for playout, storage, or processing. In step 
814. the algorithm examines whether the complete 
video frame containing the necessary video packets 
along with the synchronized audio segment is ready. If 
the video frame is ready, the process is completed (step 
815); otherwise the process is repeated by going back 
to step 514 of Figure 5 as indicated in step 81 1 . 

Step 802 shows that the video process received 
from step 708 goes to step 812, where audio segment, 
A sb (k) is adjusted to new system time S f to which a new 
video playout time has been set. In step 807, the audio 
segment is kept as a continuous reference medium for 
playout, storage, or processing although the video 
frame or packets have been dropped. 

In step 806, intermedia syochronization between 
the audio (from step 61 0 of Figure 6) and video process 
(either from step 713 of Figure 7 or from step 807) are 
performed for playout, storage or processing, and the 
processes are then sent to step 808. 

In step 808. the synchronization to the new playout 
time is initialized, since the video frame playout time has 
been changed from its original value. In step 813, the 
algorithm examines whether the complete video frame 
containing the necessary video packets along with the 
synchronized audio segment is ready. If the video frame 
is ready the process is completed (step 815); otherwise 
the process goes to step 809 where both audio and 
video processes then are reinitialized, going back to 
step 513 (Figures). 

After step 815, the processed multimedia signals 
comprising a audio segment and video frame are trans- 
mitted to the upper layer applications. In the case of 
multimedia bridge 6 (Figure 1), the audio, video and 
data signals are bridged in entities 200-1 , 200-2, and 
200-3, respectively. The bridged multimedia signals are 
then processed through entities like 203, 210, 209, 21 1 , 
213, and 214. Then the processed signals are transmit- 
ted through LAN hub 5-1, router 3-2 and ISDN WAN 5 
as previously described. A receiving LAN, such as 2-2, 
processes the audio segment and video frame signals 
as previously descrtoed in connection with Figure 3. If 
necessary, synchronization entity 104 may execute the 
algorithm shown in Figures 5-8 in order to improve per- 
formance. 

In accordance with one aspect of my invention, dif- 
ferent categories of packet mode multimedia conferenc- 
ing services can be provided: premium service, semi- 
premium service, and non-premium service. 

Referring to Figure 1 . in premium service, all con- 
ference participating end stations within a customer 
premises are considered to be connected to the 
switched nonguaranteed quality of service LANs where 
only the switched LAN hubs (e.g., hub 5-2) are shared. 
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The switched LAN hubs are directly connected to an 
ISDN WAN router (e.g., ISDN router 3-3). The traffic 
and the performance characteristics within the ISDN 
routers will satisfy the desired requirements specified in 
the premium service criteria. The ISDN WAN based 
service provider will guarantee to maintain the superior 
quality of service of the packet mode multimedia confer- 
encing services, if the shared switched hubs and ISDN 
routers operate within the given performance guide- 
lines. 

In semi-premium service mode, there can be a mix 
of switched and shared nonguaranteed quality of serv- 
ice LANs where the participating end stations are con- 
nected. In this mode, there can be a conditional 
guarantee of quality of service by the ISDN WAN based 
packet mode multimedia conferencing service provider. 

In non-premium mode, no guarantee of the quality 
of service is provided by the ISDN WAN service pro- 
vider. The service provider uses its best effort to provide 
the packet mode multimedia conferencing services. 

Those skilled in the art recognize that the preferred 
embodiments may be altered and amended without 
departing from the true spirit and scope as defined in 
the appended claims. 

Claims 

1. In a system for providing real-time multimedia con- 
ferencing services to a dispersed plurality of loca- 
tions interconnected by a wide area network from 
multimedia signals originating at said locations hav- 
ing properties adapted for said services, said loca- 
tions employing nonguaranteed quality of service 
packet switched local area networks which degrade 
at least one of said properties, said method com- 
prising in combination the steps of: 

receiving said multimedia signals from said 
local area networks at said wide area network; 
bridging said multimedia signals in said wide 
area network; and 

transmitting said bridged multimedia signals 
over said wide area network to said plurality of 
locations, whereby the bandwidth required to 
transmit said multimedia signals between said 
plurality of locations is reduced. 

2. A method, as claimed in claim 1 , and further com- 
prising the steps of: 

detecting the absence of at least said one prop- 
erty in said multimedia signals in said wide 
area network; 

processing said multimedia signals to compen- 
sate for the absence of at least said one prop- 
erty; and 

transmitting said processed multimedia signals 
to said locations over said wide area network. 
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whereby the quality of said services at said plu- 
rality of locations is improved. 

3. A method, as claimed in claim 2, wherein said step 
5 of receiving comprises the step of receiving said 
multimedia signals by sard wide area network and 
wherein said wide area network comprises an 
ISDN network. 

io 4. A method, as claimed in claim 3, wherein said step 
of transmitting comprises the step of time division 
multiplexing said processed multimedia signals for 
transmission on said ISDN network. 

is 5. A method, as claimed in claim 2, wherein said step 
of detecting comprises the steps of: 

estimating a first time period by which a first 

group of said multimedia signals is expected to 
20 be received; ^ 

determining whether any of said multimedia 

signals within said first group is absent at the 

end of said first time period; 

increasing said first time period to a second 
25 time period in the event any of said multimedia 

signals is absent; and 

determining whether any of said multimedia 
signals within said group is absent at the end of '~ 
said second time period. 

30 

6. A method, as claimed in claim 5, wherein said step - 
of detecting further comprises the step of determin- ~ 
ing whether any of said multimedia signals received 
within said first time period belong to a second 

35 group different from said first group. 

7. A method, as claimed in claim 6, wherein said step 
of processing comprises the step of deleting at 
least some of said multimedia signals belonging to 

40 said second group 

8. A method, as claimed in claim 6, wherein said step 
of processing comprises the step of storing at least 
some of said multimedia signals belonging to said 

45 second group. 

9. A method, as claimed in claim 5, wherein said step 
of processing comprises the step of creating one or 
more dummy groups of multimedia signals for said 

so first group in the event that multimedia signals 
within said first group are absent. 

10. A method, as claimed in claim 5, wherein said step 
of processing further comprises the step of syn- 

55 chronizing said multimedia signals. 

11. A method, as claimed in claim 5, wherein said mul- 
timedia signals comprise video signals and audio 
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signals and wherein said step of processing com- 
prises the step of synchronizing said video signals 
and said audio signals. 

12. A method, as claimed in claim 5, wherein said mul- 
timedia signals comprise video signals and audio 
signals and wherein said step of detecting com- 
prises the steps of: 

estimating a first time period by which a first 
segment of said audio signals is expected to be 
received; 

determining whether any of said audio signals 
within said first segment is absent at the end of 
said first time period; 

increasing said first time period to a second 
time period in the event any ol said audio sig- 
nals is absent; 

determining whether any of said audio signals 
within said segment is absent at the end of said 
second time period; 

estimating a third time period by which a first 
frame of said video signals is expected to be 
received; 

determining whether any of said video signals 
within said first frame is absent at the end of 
said third time period; 

increasing said third time period to a fourth 
time period in the event any of said video sig- 
nals is absent; and 

determining whether any of said video signals 
within said frame is absent at the end of said 
fourth time period. 

1 3. A method, as claimed in claim 1 2, wherein said step 
of detecting further comprises the steps of: 

determining whether any of said audio signals 
received within said first time period belong to a 
second segment different from said first seg- 
ment; 

determining whether any of said video signals 
received within said third time period belong to 
a second frame different from said first frame. 

14. A method, as claimed in claim 13, wherein said step 
of processing comprises the steps of: 

storing any of said audio signals belonging to 
said second segment; and 
storing any of said video signals belonging to 
said second frame. 

1 5. A method, as claimed in claim 1 3. wherein said step 
of processing comprises the step of deleting at 
least some of said video signals belonging to said 
second frame. 



1 6. A method, as claimed in claim 1 3, wherein said step 
of processing further comprises the step of syn- 
chronizing said audio signals within said first seg- 
ment with said video signals within said first frame. 

5 

17. A method, as claimed in claim 2, and further com- 
prising the steps of: 

receiving said processed multimedia signals at 
10 one of said locations over said wide area net- 

work; 

detecting the absence of at least said one prop- 
erty in said multimedia signals at said one loca- 
tion; 

is processing said one location multimedia sig- 

nals to compensate for the absence of at least 
said one property; and 

using said processed one location multimedia 
signals to create audio visual services at said 
20 one location. ^ 

18. A method, as claimed in claim 17, wherein the step 
of detecting the absence of at least said one prop- 
erty in said one location multimedia signals com- 

25 prises the steps of: 

estimating a first time period by which a first 
group of said one location multimedia signals is 
expected to be received; 
30 determining whether any of said one location 

multimedia signals within said first group is 
absent at the end of said first time period; 
increasing said first time period to a second 
time period in the event any of said one location 
35 multimedia signals is absent; and 

determining whether any of said one location 
multimedia signals within said group is absent 
at the end of said second time period. 

40 1 9. A method, as claimed in claim 1 8, wherein said step 
of detecting the absence of at least said one prop- 
erty in said received one location multimedia sig- 
nals further comprises the step of determining 
whether any of said one location multimedia signals 

45 received within said first time period belong to a 
second group different from said first group. 

20. A method, as claimed in claim 1 9, wherein said step 
of processing said one location multimedia signals 

so comprises the step of deleting at least some of said 
one location multimedia signals belonging to said 
second group. 

21 . A method, as claimed in claim 1 9, wherein said step 
55 of processing said one location multimedia signals 

comprises the step of storing at least some of said 
one location multimedia signals belonging to said 
second group. 
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22. A method, as claimed in claim 1 8, wherein said step 
of processing said one location multimedia signals 
comprises the step of creating one or more dummy 
groups of multimedia signals for said first group of 
one location multimedia signals in the event that 5 
multimedia signals within said first group of one 
location multimedia signals is absent. 

23. A method, as claimed in claim 1 8, wherein said step 

of processing said one location multimedia signals 10 
further comprises the step of synchronizing said 
one location multimedia signals. 

24. A method, as claimed in claim 1 , wherein said loca- 
tions employ switched local area network hubs and 15 
further comprising the steps of: 

sharing only said switched local area network 
hubs; and 

operating said switched local area network 20 _ 
hubs within predetermined performance guide- 
lines, whereby superior multimedia conferenc- 
ing services are provided with guaranteed 
quality of service parameters. 

25 

25. A method, as claimed in claim 1 , wherein said loca- 
tions employ a mix of shared and switched local 
area networks, whereby improved multimedia con- 
ferencing services are provided with conditional 
guarantee of quality of service parameters. 30 
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